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Abstract — A key idea in coding for the broadcast channel 
(BC) is binning, in which the transmitter encode information 
by selecting a codeword from an appropriate bin (the messages 
are thus the bin indexes). This selection is normally done by 
solving an appropriate (possibly difficult) combinatorial problem. 
Recently it has been shown that binning for the Blackwell 
channel -a particular BC- can be done by iterative schemes 
based on Survey Propagation (SP). This method uses decimation 
for SP and suffers a complexity of 0(n 2 ). In this paper we 
propose a new variation of the Belief Propagation (BP) algorithm, 
named Reinforced BP algorithm, that turns BP into a solver. 
Our simulations show that this new algorithm has complexity 
C(nlogn). Using this new algorithm together with a non-linear 
coding scheme, we can efficiently achieve rates close to the border 
of the capacity region of the Blackwell channel. 

I. Introduction 

Broadcast channels (BC) were first introduced and analyzed 
by Cover [7]. The general BC with t receivers is depicted 
in Fig. 1. In a BC, a single transmitter sends simultaneously 
independent information to multiple receivers. 

Coding for each receiver independently with a normal point- 
to-point code and sending the t messages sequentially -by al- 
locating proportions of time to each receiver- is known as time 
sharing strategy. It is shown in [7] that jointly optimized codes 
can have a larger capacity region for error-free communication 
than that of time sharing codes [1], [7], [8]. 

A key idea in coding for the BC is the binning strategy, 
which allows the transmitter to encode information by se- 
lecting a codeword from an appropriate bin. In this paper 
we deal with practical implementation of random binning 
for the BC. Existing practical binning schemes for BC are 
often based on structured codes and maximum likelihood 
algorithms. Martinian and Yedidia in [11] have used for the 
first time the random codes on graphs for quantization of a 
binary erasure source. Still their method works only for erasure 
sources and is not applicable to the general BC. 
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Fig. 1. A single sender and t receivers broadcast channel. 



Recently, Wei Yu and M. Aleksic [18] showed that the 
binning problem for a particular BC, namely the Blackwell 
Channel (BWC), when coding is performed by random low- 
density parity-check like codes, can be thought as a constraint 
satisfaction problem. They proposed an iterative encoder that 
works well at rates close to the border of the BC capacity 
region. 

The main difference of this problem with that of decoding 
classical codes is that this combinatorial problem admits many 
solutions. In fact in these cases the application of BP allows 
to compute the cardinality of the solution space but fail to find 
a particular solution. 

In [18] they use Survey Propagation (SP) algorithm for 
encoding, fixing one variable after each convergence (deci- 
mation). The main drawback of this method is the encoding 
complexity which grows as 0(n 2 ). Also the decimation works 
well only when the connectivity of XOR nodes are very small 
(c = 2, 3 and 4). 

In this paper we use a modified version of BP, called Re- 
inforced Belief Propagation (RBP), originally proposed in the 
context of perceptron learning [3], which effectively turns BP 
into a solver. Experiments show that RBP does not converge 
for factor graphs with XOR function nodes. To overcome this, 
we propose a new class of sparse non-linear codes. These two 
modifications result in a more efficient encoding complexity 
(from 0(n 2 ) to 0(n log n)) and a lower Frame Error Rate 
(FER), i.e., the probability of not finding a solution to the 



encoding problem. 

This paper is organized as follows. In the next section we 
introduce the general framework of broadcast channels and 
their capacity regions. In section III we present the iterative 
updates for BP and RBP algorithms. Our scheme for coding 
for the BWC using non-linear nodes is explained in section 
IV. Our results are presented in section V. The final section 
is devoted to conclusions and outlooks. 

II. Notations and Basic Concepts 

In this section we first introduce the basic concepts and then 
briefly review some results on capacity region for deterministic 
broadcast channels. 

Definition 2.1: A broadcast channel consists of an input 
alphabet X, two output alphabets ^1 and y 2 and a probability 
transition function P(yi,y2|x). The channel is said to be 
memoryless if 

n 

-P(yi,y2|x) = Y[P(yii,y 2 i\xi). 

i=0 

A ((2 nRl , 2 nR2 ),n) code for a BC with independent infor- 
mation consists of an encoder 

£ : 2 nRl x 2 nR2 — > X n , 

and two decoders 

£>i : y? -> 2 nRl , V 2 : -> 2" fl2 . 

We assume that the transmitted message pair (W±, W 2 ) is 
uniformly distributed over the set 2 nRl x 2 nR2 . The probability 
of error P™ is defined to be 

P e " = P(W 1 ± W x or W 2 ± W 2 ). 

Definition 2.2 (Capacity Region): A rate pair (Ri,R 2 ) is 
called achievable for the BC if there is a sequence of 
{((2 nRl ,2 nR2 ),n)} n codes with P e M as n 4 oo. The 
capacity region of the broadcast channel is the closure of the 
set of achievable rates. 

A broadcast channel is deterministic if the channel transition 
probabilities are deterministic, i.e., P(yi,y2|x) is a — 1 
function. The largest achievable rate region for a general BC 
using the binning strategy is known as the Marton's region 
[12]. This region is proved to be the capacity region for a 
discrete deterministic channels [13]. 

A well-known example of a deterministic BC is the BWC 
(see Fig. 2). The BWC has one input with three symbols and 
two outputs each one with two symbols. Given two messages 
Wi and W 2 , the goal is to find the codewords yi € 2 nRl and 
y 2 G 2 nR * such that (y u , y 2i ) ± (1, 1) for i = 1, 2, n. The 
other three combinations are allowed and they can be reached 
by selecting one of the three input symbols of the channel. 
Even though this channel is not realistic, it is a non-trivial BC 
which illustrates the conflict between transmitting information 
to first receiver and transmitting to second receiver [10]. 

Since the channel is deterministic we have H(X) = 
H(Yi,Y 2 ). In the rest we assume a uniform probability 
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Fig. 3. Rate region for the BWC with uniform distribution. 

distribution over X. With this input distribution the capacity 
region for BWC becomes 
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This capacity region is shown in Fig. 3. 

III. BP AND RBP ALGORITHMS 

Let g : S C W 1 — > M be a real valued function over the 
domain S and 

g{x 1 ,x 2 ,... 1 x n ) (x Y[ fj(Xj) (1) 

where Xj is a subset of the set of variables. 

Definition 3.1: A factor graph of a function g factorized as 
in (1) is a bipartite graph with n vertex in one part (variable 
nodes) and M vertex in the second part (factor nodes). An 
edge connects variable node Xi to factor node fj if and only 
if Xi is an argument of the local function fj, i.e., Xi € Xj. 

We show the ith marginal function associated with 
g(x!,x 2 , ...,x n ) by 

9i(xi) oc ^2 9(xi,x 2 ,-,x n ) 

where the symbol ~ {xj} indicates the set of all variable 
configurations with the i-th variable fixed to Xj. 

Calculating the marginal functions in general is a hard 
task. BP is an efficient and exact algorithm to calculate all 
marginal functions gi(xi) when the factor graph of g is cycle- 
free. It is possible to use BP also in the presence of loops. 
The resulting algorithm will be iterative and calculates the 
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Fig. 4. The modified factor graph for RBP. The black squares are dynamic 
nodes which their value is a function of the marginal of the related variable 
at a preceding iteration. 

marginals approximately. In the rest of this section, first we 
review the BP update rules and then present a generalization 
of BP called Reinforced BP algorithm (RBP) [4]. 

Let /u*_^(a;) denotes the message sent form variable node 
x to factor node / at the £th iteration. Similarly, fi^_^ x (x) 
denotes the message sent from factor node / to variable node 
x at the iteration I. Also, let 

N{xi) = {j\xi G Xj}, 
M(fj) 4 WxiCXj], 

then the BP algorithm messages can be expressed as follows: 
Local Function to Variable: 

<* e (fi( x >) n (2) 

ieM(fi)\{i} 

Variable to Local Function: 

ig^ixi) <x J] 4^ Xi (xi) (3) 

For I = 1, we initialize the messages fi x ^f(x) randomly. 
These updating rules tell us how to produce locally outgoing 
messages from incoming messages. We define the marginal 
function of variable Xi at iteration I + 1 as 

The algorithm converges after t iterations if and only if for 
all variables Xj and all function nodes fj 

In practice we need to predefine maximum number of iter- 
ations £ max and a precision parameter e as the input to the 
algorithm. 

BP has been generalized/modified in many ways [2], [4], 
[5], [15], [19]. BP and its generalizations have proven to 
be efficient when the variables are biased toward a solution. 
Unfortunately when this condition is not fulfilled marginal 
themselves are not sufficient to find a solution to the com- 
binatorial problem and one has to resort to some decimation 



techniques ( [2], [18]), resulting in a high computational 
complexity. 

We will show here the RBP equations [3] that turn BP 
into an efficient solver. The idea is to introduce a new set 
of reinforcement messages which drive the equations toward 
a single solution. First we modify the original factor graph by 
adding to each variable node a new function node. In Fig. 4 
these new function nodes are depicted by black squares. These 
function nodes are dynamic and at the tth iteration take the 
value (gf^ixi)) 1 ^ 1 \ i.e, a power of the marginal of the 
variable Xi at the preceding iteration, ^(t) is a non decreasing 
function in [0, 1] with 7(0) = 0. While the updating rule (2) 
at function nodes does not change for RBP, the variable to 
function messages should be modified as below. 

Variable to Local Function for RBP: 

In this paper we deal only with binary constraint satisfaction 
problems, where x £ {0, 1}" and the local functions fj(Xj) 
are 0-1 indicator functions. A vector (xi, X2, ■ x n ) satisfies 
fj(Xj) if fj(Xj) = 1. (xi, X2, ■ x n ) is called a solution of 
the constraint satisfaction problem if all local functions are 
satisfied, i.e., Ylj^M fj(Xj) = 1- It is easy to show that if 
RBP converges, it converges to a solution of our problem (all 
messages completely polarized to delta functions). This simple 
modification provides us with a solver with complexity 0(n) 
(assuming roughly constant convergence time). Note that the 
number of iteration of RBP depends also on the choice of -y(£) 
in (5). As our experiments show, choosing an optimal 7 can 
dramatically decrease the number of iterations of RBP at least 
for the binning problem. For the rest of this paper we will set 

7(f) = 1 - 7 o7i , (6) 
where 70, 71 are in [0, 1]. 

IV. Coding for the Blackwell Channel Using 
Non-Linear Nodes 

One of the main coding strategies for deterministic broad- 
cast channel is binning. The idea is to generate 2 nH ( Yl ^ 
codewords yi and 2 nH ^ codewords y 2 and randomly assign 
them into 2 nRl and 2 nR2 bins. To transmit a particular pair of 
bin indices (i, j), the transmitter looks for a pair of codeword 
(yi,y2) £ such that they are jointly typical. 

For the BWC, the joint typicality of yi and y2 is equivalent 
to being consistent with the channel constraints. Therefore, we 
are looking for efficient ways to finding a pair (yi,y2) such 
that (yu, yii) = (1,1) does not occur for i = 1, 2, n. 

Wei Yu and Marko Aleksic in [18] have suggested a 
random binning method for BWC based on low-density parity- 
check like codes. In this section we first review their results 
and then modify their scheme using non-linear nodes and 
RBP algorithm. As we will see in the next section, these 
modifications imply a better encoding complexity and a lower 
FER for large function node connectivity. 
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Fig. 5. Factor graph for LDPC like encoding for Blackwell channel. 

Fig. 5 illustrates the graphical structure (factor graph) of 
the coding scheme used in [18]. n circles denote the variable 
nodes, nRi + nRi squares denote the parity check nodes and 
n crossed squares denote the product constraints (ensuring the 
(1, 1) pair does not occur). The encoding process is as follows. 
The information bits nRi and nR-2 are placed at the parity 
checks. These values are actually the bin indices. The goal is 
to find the set of variable assignments that satisfy the parity- 
checks and product constraints simultaneously. These two sets 
of constrains ensure the typicality of the pair (yi,y2)- When 
n is large an exhaustive search is not feasible and practical 
algorithms are desirable. 

In [18] the survey propagation algorithm is suggested for 
this encoding problem. The main drawback of using SP/BP 
is the complexity which grows as 0(n 2 ) because of the 
decimation process. As it was also mentioned in [18], this 
method works only for small function nodes connectivity. 

On the other hand, the RBP algorithm, introduced in the 
section III, do not converge -even for rates not close to the 
capacity- for linear codes. To overcome this, we substitute 
parity check nodes with non-linear (random) functions. These 
kind of gates have been analyzed with methods from statistical 
physics [6]. Intuitively, the reason for which random gates 
may show a better performance with respect to the linear 
nodes can be explained as follows. Strong symmetry properties 
of XOR functions do not allow a decimation procedure to 
choose a good decimation path that preserves the uncorrelation 
hypothesis needed for BP; indeed, in any decimation step with 
XOR gates, undecided variables have all equal probability of 
taking or 1. 

Given c variable input nodes we choose a non-linear func- 
tion node randomly from all 2 C possible balanced truth-tables. 
We eliminate from this choice fully-canalized nodes, i.e., 
random nodes for which a particular value of one of their 
variables determine the output. For our code constructions we 
have used 4 to 8 different random nodes for each connectivity 
c. Note that the complexity of updating messages on a random 
node with degree c is of order 2 C . In this paper we confine 
ourselves to a constant degree c = 6 and hence ignore this 
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Fig. 6. Entropy as a function of rate (iti = R2 ) for different function node 
connectivity c. At any given rate the entropy of codes with non-linear factor 
nodes increases with c and approaches the entropy of linear codes. 
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0.00011 
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0.0119 


0.0347 


71 


0.99 


0.995 


0.999 


0.9995 


0.9999 


0.99999 


0.999995 



TABLE I 

BER AND FER OF NON-LINEAR LDPC LIKE ENCODERS AT A GIVEN RATE 
(Rl = R2) AND CONNECTIVITY C = 6. 



factor in the rest. 

In order to show the suitability of non-linear nodes to 
the problem at hand we compute the normalized size of the 
solution space, defined as H$ = \og(N s ) /n, where N s is 
the number of solutions. An approximation to H$ can be 
computed directly from the BP messages at a fixed point [19]. 

In Fig. 6 we plot Hs as a function of rate using linear nodes 
and non linear nodes for different values of the function node 
connectivity c. The entropy of codes with non-linear function 
nodes increases with c and approaches the entropy of linear 
codes. Note that for linear codes the entropy does not change 
with connectivity. A connectivity c = 6 thus guarantees a 
solution space with cardinality near to those of LDPC codes 
when using non-linear type nodes. This value of connectivity 
has then been chosen for the code construction. 

V. Results 

Table I shows the FER and BER of our constructed non- 
linear codes for the BWC with n = 1000 and constant 
connectivity c = 6 at different rates. The last line reports the 
values we chose for 71 . 

We estimated the algorithmic complexity of the presented 
coding scheme in a series of experiments described below. In 
particular, we will show how the convergence time changes as 
a function of n and 71 . The RBP algorithm was run with an 
estimated optimal value of 71, and we have chosen a cutoff 
time of tt^ — r to measure the bit and frame error rates. 

(!-7l) 

Fig. 7 shows the average number of iterations needed (in the 
case of success) for a rate Ri = R2 = 0.70 as a function of 
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Fig. 7. The average number of needed iterations as a function of 71 at rate 
Rl = R2 = 0.70 for different values of n. Note that for smaller 71 we need 
less number of iterations but both BER and FER are larger (see Fig. 8 and 
table I). 



DC 
LU 
CD 




0.993 



0.995 



0.997 



0.999 



Yi 



Fig. 8. Bit error rate as a function of 71 at rate Ri = R2 = 0.70 for 
different values of n. Error bars are smaller than symbols size in this scale. 



n and 71 and for 160 encoding operations. These simulations 
indicate that the number of iterations increase as O(logn). 
Although the number of iterations increase (exponentially) 
with 71 , both the BER and FER decrease (exponentially) as it 
can be seen in Fig. 8. Note that for rates closer to the capacity 
bound (Ri = i?2 ~ 0.785) a value of 71 closer to 1 (and larger 
number of iterations) is needed. 

Although the results depicted in Fig. 7 indicate a logarithmic 
increase in the number of iterations as a function of n, this 
result may be due to a not optimized choice of 7(£). For 
example by choosing 70 = 0.8 in (6) it is possible to reduce 
the number of iterations for n = 4000 and 71 = 0.999 by 
nearly 25%. In other words, one can avoid approximately the 
first 200 iterations of RBP without loosing in performance. 

VI. Conclusion and Outlooks 

We have introduced a novel variation of the BP algorithm, 
called reinforced BP, that turns it into an efficient solver for 



non-linear problems even when they have a large solution 
space. The algorithm have the same complexity of BP and thus 
considerably smaller than the decimation approach applied to 
BP/SP proposed in [18]. 

Using RBP we have constructed a general and rather 
efficient encoding scheme for the BWC. Our codes provide 
good encoding performances for rates up to 0.72. This result 
can be possibly improved by optimizing the function and 
the degree distributions of the code. 

Our scheme compares well with existing ones: for linear 
codes with R = Ri = R-2 = 0.75 and decimation, as it was 
reported also in [18], one can get the bit error rate of 5.10 -3 . 
Still, simulations show that the FER in this case is 0.9 and 
it does not improve for smaller rates like R = 0.72 with the 
same connectivity. On the other hand it works only for low 
function node connectivity. Our scheme is much more flexible 
and provides a comparable FER and BER at R = 0.75 with 
lower computational complexity. For smaller rates our codes 
outperform the existing linear encoding schemes. 
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